Fusing camera with LiDAR is a promising technique to improve the accuracy of 3D detection due to the complementary physical properties. While most existing methods focus on fusing camera features directly with raw LiDAR point clouds or shallow 3D features, it is observed that direct deep 3D feature fusion achieves inferior accuracy due to feature misalignment. The misalignment that originates from the feature aggregation across large receptive fields becomes increasingly severe for deep network stages. In this paper, we propose PathFusion to enable path-consistent LiDAR-camera deep feature fusion. PathFusion introduces a path consistency loss between shallow and deep features, which encourages the 2D backbone and its fusion path to transform 2D features in a way that is semantically aligned with the transform of the 3D backbone. We apply PathFusion to the prior-art fusion baseline, Focals Conv, and observe more than 1.2\% mAP improvements on the nuScenes test split consistently with and without testing-time augmentations. Moreover, PathFusion also improves KITTI AP3D (R11) by more than 0.6% on moderate level.
translated by 谷歌翻译
Diffusion models have emerged as a powerful tool for point cloud generation. A key component that drives the impressive performance for generating high-quality samples from noise is iteratively denoise for thousands of steps. While beneficial, the complexity of learning steps has limited its applications to many 3D real-world. To address this limitation, we propose Point Straight Flow (PSF), a model that exhibits impressive performance using one step. Our idea is based on the reformulation of the standard diffusion model, which optimizes the curvy learning trajectory into a straight path. Further, we develop a distillation strategy to shorten the straight path into one step without a performance loss, enabling applications to 3D real-world with latency constraints. We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods. On real-world applications such as point cloud completion and training-free text-guided generation in a low-latency setup, PSF performs favorably.
translated by 谷歌翻译
Accurate segmentation of power lines in aerial images is essential to ensure the flight safety of aerial vehicles. Acquiring high-quality ground truth annotations for training a deep learning model is a laborious process. Therefore, developing algorithms that can leverage knowledge from labelled synthetic data to unlabelled real images is highly demanded. This process is studied in Unsupervised domain adaptation (UDA). Recent approaches to self-training have achieved remarkable performance in UDA for semantic segmentation, which trains a model with pseudo labels on the target domain. However, the pseudo labels are noisy due to a discrepancy in the two data distributions. We identify that context dependency is important for bridging this domain gap. Motivated by this, we propose QuadFormer, a novel framework designed for domain adaptive semantic segmentation. The hierarchical quadruple transformer combines cross-attention and self-attention mechanisms to adapt transferable context. Based on cross-attentive and self-attentive feature representations, we introduce a pseudo label correction scheme to online denoise the pseudo labels and reduce the domain gap. Additionally, we present two datasets - ARPLSyn and ARPLReal to further advance research in unsupervised domain adaptive powerline segmentation. Finally, experimental results indicate that our method achieves state-of-the-art performance for the domain adaptive power line segmentation on ARPLSyn$\rightarrow$TTTPLA and ARPLSyn$\rightarrow$ARPLReal.
translated by 谷歌翻译
Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.
translated by 谷歌翻译
由于经过验证的2D检测技术的适用性,大多数当前点云检测器都广泛采用了鸟类视图(BEV)。但是,现有方法通过简单地沿高度尺寸折叠的体素或点特征来获得BEV特征,从而导致3D空间信息的重丢失。为了减轻信息丢失,我们提出了一个基于多级特征降低降低策略的新颖点云检测网络,称为MDRNET。在MDRNET中,空间感知的维度降低(SDR)旨在在体素至BEV特征转换过程中动态关注对象的宝贵部分。此外,提出了多级空间残差(MSR),以融合BEV特征图中的多级空间信息。关于Nuscenes的广泛实验表明,该提出的方法的表现优于最新方法。该代码将在出版时提供。
translated by 谷歌翻译
我们提出了一个首次击中扩散模型(FHDM)的家族,该模型是深层生成模型,该模型以扩散过程生成数据,该过程在随机的首次击中时间终止。这产生了在预先指定的确定性时间终止的标准固定时间扩散模型的扩展。尽管标准扩散模型是为连续不受约束的数据而设计的,但FHDM自然设计用于在连续以及一系列离散和结构域上学习分布。此外,FHDM启用依赖实例的终止时间,并加速扩散过程,以更少的扩散步骤采样更高质量的数据。从技术上讲,我们通过根据DOOB的$ h $转换得出的有条件的首次击中过程(即桥)来训练FHDM,以最大的似然估计从观察到的数据增强的扩散轨迹(即桥梁),从而偏离了常用的使用时间反转机制。我们应用FHDM在各个领域中生成数据,例如点云(一般连续分布),地球上的气候和地理事件(球体上的连续分布),未加权图(二进制矩阵的分布)以及2D图像的分割图(高度图像(高) - 二维分配)。我们观察到与质量和速度的最新方法相比,相比之下。
translated by 谷歌翻译
基于AI的分子生成为大量生物医学科学和工程(例如抗体设计,水解酶工程或疫苗开发)提供了一种有希望的方法。由于分子受物理定律的管辖,所以关键的挑战是将先前的信息纳入训练程序中,以产生高质量和现实的分子。我们提出了一种简单而新颖的方法,以引导基于扩散的生成模型培训具有物理和统计的先验信息。这是通过构建物理知情的扩散桥,即保证在固定末端产生给定观察的随机过程来实现的。我们开发了一种基于Lyapunov函数的方法来构建和确定桥梁,并提出了许多有关高质量分子生成和均匀性促进的3D点云生成的信息丰富的先验桥的建议。通过全面的实验,我们表明我们的方法为3D生成任务提供了强大的方法,从而产生具有更好质量和稳定性得分的分子结构,并且具有更高质量的分布点云。
translated by 谷歌翻译
基于扩散的生成模型最近取得了令人鼓舞的结果,但在概念理解,理论分析,算法改进和扩展到离散,结构化的,非欧盟域的扩展方面提出了一系列开放问题。这项工作试图重新研究整体框架,以获得更好的理论理解并为来自任意域的数据开发算法扩展。通过将扩散模型视为具有未观察到扩散轨迹的潜在变量模型,并应用最大的似然估计(MLE),并用辅助分布估算的潜在轨迹,我们表明,潜在轨迹的模型构建和插入的潜在轨迹构成了构建扩散桥的过程,从而实现了扩散桥梁的过程终点的确定性价值和约束,为此我们提供了系统的研究和一套工具。利用我们的框架,我们提出了1)对学习扩散生成模型的第一个理论错误分析,以及2)一种简单而统一的方法,用于从不同离散和受限域中学习数据。实验表明,我们的方法在生成图像,语义片段和3D点云方面表现出色。
translated by 谷歌翻译
昼夜节律的破坏是阿尔茨海默氏病(AD)患者的基本症状。人类脑中基因表达的完整昼夜节律编排及其与AD的固有关联仍然很大程度上是未知的。我们提出了一种新颖的综合方法,即Prime,以检测和分析在多个数据集中不合时宜的高维基因表达数据中的节奏振荡模式。为了证明Prime的实用性,首先,我们通过从小鼠肝脏中的时间课程表达数据集作为跨物种和跨器官验证来对其进行验证。然后,我们将其应用于研究来自19个对照和AD患者的19个人脑区域的未接收基因组基因表达中的振荡模式。我们的发现揭示了15对控制大脑区域中清晰,同步的振荡模式,而这些振荡模式要么消失或昏暗。值得注意的是,Prime在不需要样品的时间戳而发现昼夜节律的节奏模式。 Prime的代码以及在本文中复制数字的代码,可在https://github.com/xinxingwu-uk/prime上获得。
translated by 谷歌翻译
由于异质访问点(APS)的性质,负载平衡(LB)是混合灯保真度(LIFI)和无线保真度(WIFI)网络(HLWNETS)的挑战性问题。机器学习有可能以近乎最佳的网络性能为培训过程提供复杂性的LB解决方案。但是,当网络环境(尤其是用户数量)更改时,需要进行最先进的(SOTA)学习辅助LB方法,这大大限制了其实用性。在本文中,提出了一个新颖的深神经网络(DNN)结构,称为自适应目标条件神经网络(A-TCNN),该结构在其他用户的条件下为一个目标用户进行AP选择。此外,开发了一种自适应机制,可以通过分配数据速率要求将较大数量的用户映射到较大的数字,而不会影响目标用户的AP选择结果。这使提出的方法可以处理不同数量的用户,而无需再进行重新培训。结果表明,A-TCNN实现了非常接近测试数据集的网络吞吐量,差距小于3%。还证明,A-TCNN可以获得与两个SOTA基准相当的网络吞吐量,同时最多将运行时降低了三个数量级。
translated by 谷歌翻译